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Abstract 

We compute spectra and Brown measures of some non self-adjoint operators in 
(M2(C),irr) * (M2(C),irr), the reduced free product von Neumann algebra of 
M2(C) with M2(C). Examples include AB and A + B, where A and B are matrices 
in (M2(C), iTr) * 1 and 1 * {A'hiC), ^Tr), respectively. We prove that AB is an 
R-diagonal operator (in the sense of Nica and Speicher [12]) if and only if Tr{A) = 
Tr{B) = 0. We show that \i X = AB or X = A + B and A,B are not scalar 
matrices, then the Brown measure of X is not concentrated on a single point. By 
a theorem of Haagerup and Schultz [9j, we obtain that \i X = AB oi X = A + 
B and X ^ Al, then X has a nontrivial hyper invariant subspace affiliated with 
(M2(C),irr)*(Af2(C),irr). 

Keywords: free products, spectrum. Brown measure, R-diagonal operators, hyperinvari- 
ant subspaces 

1 Introduction 

In 1983, L.G. Brown [2] introduced a spectral distribution measure for non-normal elements 
in a finite von Neumann algebra with respect to a fixed normal faithful tracial state, which 
is called the Brown measure of the operator. Recently, U. Haagerup and H. Schultz [9] 
proved a remarkable result which states that if the support of Brown measure of an opera- 
tor in a type IIi factor contains more than two points, then the operator has a non-trivial 
hyperinvariant subspace affiliated with the type IIi factor. In general cases, the computa- 
tion of Brown measures of non-normal operators are nontrivial. The first essential result 
was given by Haagerup and F. Larsen. In [8j, Haagerup and Larsen computed the spectrum 
and Brown measure of R-diagonal operators in a finite von Neumann algebra, in terms of 
the distribution of its radial part. Brown measures of some non- normal and non- R-diagonal 
operators, examples include m„ + u^, where u„ and the generators of Z„ and Z 

respectively, in the free product Z„ * Z, and elements of the form + iSp, where and 
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S/3 are free semi-circular elements of variance a and (3, are computed by P. Biane and 
F. Lehner in [T]. The purpose of this paper is to compute the spectra and Brown measures 
of some non hermitian operators in (M2(C), ^Tr) * (M2(C), ^Tr), the reduced free product 
von Neumann algebra of M2(C) with M2(C) (cf. [Ch]). Examples include AB and A + B, 
where A and B are matrices in (M2(C), |Tr) * 1 and 1 * (M2(C), |Tr), respectively. This 
paper is organized as follows. 

In section 2 we recall preliminary facts about Brown measures, R-diagonal operators, 
Haagerup and Larsen's result on Brown measures of R-diagonal operators and some nota- 
tion used in this paper. In section 3, we provide some results on the spectra and spectral 
radius of operators in M2(C) *M2(C), the universal free product C*-algebra of M2(C) with 
M2(C). Firstly we compute the spectral radius of AB for two normal matrices A G M2(C)*1 
and B E 1 * M2(C) relative to M2(C) * M2(C). As a corollary, we also get the spectrum 
radius of AB for normal matrices A G (M2(C), |Tr) * 1 and B E 1* (M2(C), |Tr), relative 
to the reduced free product von Neumann algebra of M2(C) with M2(C). Then we obtain 
the following result: Let A,B be matrices in M2(C)*1 and 1*M2(C), respectively, such that 
Tr{A) = Tr{B) = 0. Then a{AB), the spectrum of AB, relative to M2(C) * M2(C), is the 
closure of the annulus centered at with inner radius and outer radius 

\\A\\ \\B\\, where we use the convention oo"^ = and if A is not invertible then HA^""^!! := c>o. 

In section 4 we prove that AB is an R-diagonal operator if and only if Tr{A) = Tr{B) = 
0, where A e (M2(C), ^Tr) * 1 and S e 1 * (M2(C), |Tr). As a corollary, we exphcitly 
compute the spectrum and Brown measure of AB {Tr{A) = Tr{B) = 0) in terms of 
§— transform of A* A and B*B. 

In section 5, we develop algebraic techniques used in [4j. Let X E 1* (M2(C),|Tr). 

With respect to the matrix units of (M2(C),iTr)*l,X = ( . By [4J, (M2(C),|Tr)* 

(M2(C), iTr) = L(F3) ® M2(C). So xi,X2,X3,X4 G /.(Fa). In section 5, we find *-free gen- 
erators h,u,v of LiWs) (different from the free generators given in [4]) so that we may 
explicitly write out Xi, X2, X3, X4 in terms of h, u, v. 

In section 6, we compute miscellaneous examples of Brown measures of operators A + B 
and AB, where A e (M2(C), ^Tr) * 1 and 5 G 1 * (Af2(C), ^Tr). As a corollary, we show 
that A + B is a.n R-diagonal operator if and only if A + B = 0. 

In section 7, we prove the following result: Let A G (M2(C), |Tr) * 1 and B G 
1 * (M2(C),|Tr). iiX = A + BoTX = AB and A,B are not scalar matrices, then 
the Brown measure of X is not concentrated on a single point. As a corollary of Theorem 
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7.1 of [H-Sl], we prove that if X = A + B or X = AB and X ^ Al, then X has a nontrivial 
hyperinvariant subspace affihated with (M2(C), \Tr) * (M2(C), \Tr). 

Many concrete examples of spectra and Brown measures are given in this paper. For 
some interesting apphcations, we refer to |5]. 



2 Preliminaries 

2.1 Fuglede-Kadison determinant and Brown's spectral measure. 

Let be a finite von Neumann algebra with a faithful tracial state r. The Fuglede-Kadison 
determinant [6], A : ^ [0, cxd[, is given by 



with exp{— oo} := 0. For an arbitrary element T in AI the function A — log A(a — Al) is 
subharmonic on C, and it's Laplacian 



in the distribution sense, defines a probability measure ht on C, called the Brown's mea- 
sure [2] of T. From the definition, Brown measure fiT only depends on the joint distribution 
of T and T* . 

If T is normal, ht is the trace r composed with the spectral projections of T. If 
Al = M„(C) and r = -Tr is the normalized trace on M„(C), then ht is the normalized 
counting measure ^ {5\^ + '^A2 " " " + ^a„), where Ai, A2 ■ ■ ■ , A„ are the eigenvalues of T re- 
peated according to root multiplicity. 

The Brown measure has the following properties (see [21 HD]): /^r is the unique com- 
pactly supported measure on C such that logA(T-Al) = j^\og\z-\\d^iT{z) for all A G C. 
The support of is contained in cr(T), the spectrum of T. ^st = ^^TS for arbitrary S,T 
in Ai, and if f{z) is analytic in a neighborhood of cr{A), fif(T) = (a^t)/, the push-forward 
measure of fix under the map A /(A). If G Al is a projection such that E G LatT, 
then with respect to E,I — E we can write 



A(T)=exp{r(log|T|)}, 



Tg Al 



dfirW :=:^v'fogA(r-Al), 
Zn 
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where A = ETE and C = {I - E)T{I - E) are elements of Mi = EME and M2 = 
(/ — E)M.[I — E), respectively. Let /i^ and /ic be the Brown measures of A and C com- 
puted relative to A^i and Ai2, respectively. Then /i^ = (1 — a)fic, where a = t{E). 

For a generalization of Brown measures of sets of commuting operators in a type IIi 
factor, we refer to \15\ . 

2.2 R-diagonal operators 

In 1995, A. Nica and S. Speicher [12] introduced the class of R-diagonal operators in non- 
commutative probability spaces. Recall that an operator T in a non-commutative proba- 
bility space is an i?— diagonal operator if the i?— transform Rfj,(T,T*) of the joint distribution 
/x(T, T*) of T, T* is of the form 

OD 00 

n=l n=l 

Nica and Speicher [12] proved that T is an i?— diagonal operator if and only if T has same 
*-distribution as product UH, where U and H are *-free random variables in some tracial 
non commutative probability space, f/ is a Haar unitary operator and H is positive. If 
T is an R-diagonal operator, then the *-distribution of T is uniquely determined by the 
distribution of T*T = |Tp. If T is an R-diagonal operator and S is *-free with T, then 
both ST and TS are R-diagonal operators (see [12]). If T is an R-diagonal operator and 
G N, then T" is also an R-diagonal operator (see [HITT]). For other important properties 
of R-diagonal operators, we refer to [H [TTl [121 [13] • 

2.3 Brown measures of R-diagonal operators 

In [8] , Haagerup and Larson explicitly computed the Brown measures of R-diagonal oper- 
ators in a finite von Neumann algebra. 

Theorem 2.1. (Theorem 4.4 of [8J) Let U,H be *-free random variables in a noncommu- 
tative probability space {Ai,T), with U a Haar unitary operator and H a positive operator 
such that the distribution fi^ of H is not a Dirac measure. Then the Brown measure Huh 
of UH can be computed as the following. 

1. Huh is rotation invariant and its support is the annulus with inner radius ||-f^^^||2^ 
and outer radius ||i^||2- 

2. /if///({0}) = /iH({0}) andforte]iJiH{{^})A], 

flUH (b (0, (§^^,(t-l))~'/')) =t. 
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where §^^2 ^-^ the ^—transform of and B(0,r) is the open disc with center and 
radius r; 

3. fiuH is the only rotation invariant symmetric probability measure satisfying 2. 

Furthermore, if H is invertihle, then a{UH) — supp/iun,' if H is not invertible, then 
a{UH) = B(0, \\H\\2). 

2.4 Some Notation 

The following notation will be used in the rest of the paper 



{M,t) = (M2(C), |Tr) * (M2(C), ^Tr) denotes the reduced free product von Neu- 
mann algebra of M2(C) with M2(C) with the unique tracial state r; 



• M2(C)(i) := (M2(C), |Tr) * 1 and Af2(C)(2) := 1 * (M2(C), |Tr); 

• {Eij}i j=i 2, {Fij}i j=i^2 are matrix units of M2(C)(i) and M2 (C)(2), respectively; 

• P^Eii and Q = Fu] 

. M ^ N ® M^iC)^^) ^ N ® M2iC)^2)- For X e At, X = ( = 

fx' x' \ 

{ j J \ means the decomposition is with respect to above matrix units of 

V ^3 ^4 / (2) 

-^2 (C)(1) and M2 (C)(2), respectively. 



• A,Ai,--- ,An denote elements in M2(C)(i), B, Bi, • • • , Bn denote elements in M2(C)(2), 
X, y, Z denote general elements in A4; 

• An element X in Al is called centered if t{X) — 0. 

We end this section with the following lemma. The proof is an easy exercise. 
Lemma 2.2. V"iM2(C)(i)yi is free with M2{C)^r). 
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3 Spectra of elements in the universal free product of 

M2(C) and M2(C) 

Let A = M2(C)*M2(C) denote the universal free product C*-algebra of M2(C) with M2(C). 
Then there is a * homomorphism tt from A onto the reduced free product C*-algcbra of 
M2(C) and M2(C), the C*-subalgebra generated by M2(C)(i) and M2 (C)(2) in M. Since 
a{7r{a)) C a{a) for a G A, it is useful to obtain some information of spectrum oi AB, where 
A e M2(C) * 1 and 5 e 1 * M2(C). 



3.1 "EVee product" of normal matrices 

Lemma 3.1. Let A e M2(C) * 1 and B & 1 * M2(C) be normal matrices. Then r{AB) — 
\\A\\ ■ \\B\\ relative to A. 

Proof. r{AB) < \\AB\\ < \\A\\ ■ \\B\\. We need only to prove r{AB) > \\A\\ ■ \\B\\. Since 
A is a normal matrix, there is a unitary matrix Ui € M2(C) * 1 such that UiAU^ = 
ai 



P: 



and 1 1 Oil II — \\A\\. Similarly, there is a unitary matrix U2 ^ 1 * M2(C) such 



that U2BU^ = yo p J = ll^ll- Let 7ri(X) = UiXU^ and 7r2(r) = U2YU^ 

be ^-representations of Mi(C) * 1 and 1 * M2(C) to M2(C), respectively. Then there is a 
*-representation tt = tti * 712 from A to M2(C) and 7r(Ai3) = /a /a ) ■ Therefore, 



P1P2 

aia2 e a{7r{AB)) C a{AB). So r(AS) > |q;iq;2| = ||^|| • ||S||. □ 

Corollary 3.2. Let A e M2(C)(i) and B e M2 (C)(2) be normal matrices. Then r{AB) = 
\B\\ relative to Ai. 



Proof. We may assume that A and B are diagonal matrices. Then we can treat AB as an 
operator in the full free product C*(Z2 * Z2). Same technique used in the previous lemma 
gives the corollary. □ 



3.2 "Free product" of non-normal matrices 

It is well-known that two matrices X,Y in M2(C) are unitarily equivalent if and only if 
Tr{X) = Tr{Y),Tr{X'^) = TriV^) and Tr{X*X) = Tr{Y*Y). The proof of the following 
lemma now is an easy exercise. 

Lemma 3.3. If A & M2(C) and Tr{A) — 0, then A is unitarily equivalent to a matrix of 

form ( |i n ) ' "^here a, (3 are complex numbers. 
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Remark 3.4. We have the following useful observations: 

iWO a \ f l\_/0 p ' 

1 J \ (3 J \1 J ~ \ a 

e^(ei-e2)/2 ^ i^igie^ J V e-'(^i-^2)/2 ^ - 

Lemma 3.5. Let A G Af2(C) * 1 and G 1 * M2(C) 6e matrices such that Tr{A) = 
Tr{B) = 0. Then r{AB) = \\A\\ ■ \\B\\ relative to A. 

Proof. We need only to prove r(Ai?) > By Lemma 3.3 and Remark 3.4, there arc 

unitary matrices U, V in M2(C) such that UAU* = ( ^' and VBV* ^ ' ^ "2 



A ; " ^ ' ' V 

and = l/^a] = Let 7ri(X) = [/XC/* and 7r2(y) = VYV* be *-representations 

of M2(C) * 1 and 1 * M2(C) to M2(C), respectively. Let vr = tti * 7r2 be the induced *- 
representation of 1 to M2(C). Then (t(A5) D a{Ti{AB)) = a{7ri{A)TT2{B)) = {ai/52, a2pi}. 
Therefore, r{AB) > \ai(32\ = \\A\\ ■ \\B\\. □ 

Theorem 3.6. Let A G M2(C) * 1 and B e 1 * M2(C) 6e matrices such that Tr{A) = 
rr(5) = 0. Then 

a{AB) = [\\A-'\\-'\\B-'\\-\ \\A\\\\B\\] x, [0,27r], 

where Xp denotes the polar set product {re*^ : r G [||74~^||~^||i?~^||~^, ||74||||i?||], 9 G 
[0,27r]}. 

Proof. We will prove the theorem for two cases. 

Case 1. Either A or B is not invertible. We may assume that A is not invertible. By 
Tr{A) = 0, Lemma 3.3 and Remark 3.4, A is unitarily equivalent to ^ ^ 'o^ ^ ' ^^^^"^^^ 

loss of generahty, we assume that A = ^ ^ } ^ -^2(C) * 1. By Lemma 3.3 and Remark 

3.4, we may also assume that 5=^^ ^^Gl* M2(C) and (5 > a > 0. We need to 

prove that a{AB) is the closed disc of complex plane with center and radius f3. Since A 
is unitarily equivalent to e'^^A in M2(C) * 1, a{AB) is rotation invariant. For 9 G [0,27r], 

let U — ( sin 9 \ niiX) — X and n2(Y) — UYU* be *-representations 

\^ — sm6' cosb* J 

of M2(C) * 1 and 1 * M2(C) to M2(C), respectively. Let tt = tti * 7r2 be the induced 
*-representation of A to M2(C). Then 

A n\ attt:>tt* f — a sin^ 9 + P cos^ 9 —(a + /3)sin9cos9 
TT{AB) = AUBU = I ^ ^ 
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So a{7i{AB)) = {-asin^e + Pcos'^9,0}. Since [0,/?] C [-a,P] = {-asin^ 9 + P cos^ 9 : 
9 e [0,27r]}, [0, /3] C a{AB). Since a{AB) is rotation invariant, a{AB) contains the closed 
disc with center and radius p. By Lemma 3.5, a{AB) is the closed disc of complex plane 
with center and radius (3. 



Case 2. Both A and B are invertible. By Lemma 3.3 and Lemma 3.4, we may assume that 
and B = ^ such that pi, p2 > 1- Then ^"^ = ° ^ and 

= 1^ J ^2 ^ . We need to prove that a{AB) = [1, P1P2] Xp [0,27r]. By Lemma 3.5, 

r{AB) = P1P2 and r{{AB)-^) = 1. This implies that a{AB) C [l,/3i/?2] Xp [0,2%]. So we 
need only to prove a{AB) 3 [l,/9i/32] Xp [0,2%]. 

For 4',il> E [0,27r], let U — ^ *^shi^ e*''^cos^ '^^^^ C/ is a unitary matrix. Let 

7ri(X) = C/XC/* and 7r2(F) = F be *-representations of M2(C) * 1 and 1 * M2(C) to M2(C), 
respectively. Let tt = tti * 772 be the induced *-representation of A to M2(C). Then 

M - -/5i/?2e''^ sin^ ^ + /52e-^'^ cos^ ijj * 

* /^le^'^cos^V'-e-^'^sin^V' 

Let Ai(0, '0), X2{4>, ip) be the eigenvalues of 7:{AB). Then 

Ai(0, V)A2(0, ip) = det(7r(AS)) = det(A) det(S) = /3i/32, (3.1) 

Ai(0, ij) + A2(0, ^) = iPie"^ + /92e-''^) cos' ^ - iPip2e''^ + e"'^) sin' ^/;. (3.2) 

Note that a(Afi) D {Ai(0,^) : (j),^ e [0,27r]}. We only need to prove that {Ai(0,^) : 
(l),ip e [0, 2%]} I) [1, P1P2] Xp [0, 2%]. For this purpose, we need to show for any r e [1, /9i/32], 
^ e [0, 27r], there are 0, £ [0, 27r] such that 

re^^ + (^e-'^ = (^e'* + p2e-''^) cos' - (/3i/?2e''^ + e'"'^) sin' V'- (3.3) 
r 

Let q; = cos'V'. Simple computations show that equation 3.3 is equivalent to the following 

^PlP2\ .^.f P1P2. . , 

r H cos 9 + I \ r sm 9 — 



r 

{a{l + /3i)(l + P2) - (1 + P1P2)) COS0 + i{a{P, - 1){P2 + 1) + (1 - P1P2)) sin0. 
Let 

n^ = S(r + ^] cos9 + t(r- ^] sm9 : r G [1, PiP2] , 9 e [0, 2%] ] , 
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1^2 = {(a(l + + P2) - (1 + PM) COS0 + liaifh - l)(/32 + 1) + (1 - P1P2)) sin0 

: a e [0,1], e [0,27r]}. 

Now we need only to prove fli = fl2. Note that fli is the union of a family of ellipses with 
center the origin point and semimajor axis and semiminor axis |r + and |r — ^^|, 
1 < r < /3i/32, respectively. Similarly, Q2 is the union of a family of ellipses with center the 
origin point and semimajor axis and semiminor axis \a{l + + ^2) — (1 + /9i/?2)| and 

- l)(/32 + 1) + (1 - PiP2)\, < a < 1, respectively. Note that the "largest" ellipse 
in Qi is with semimajor axis and semiminor axis |1 + (3if32\ and \(3il32 — 1|, respectively; 
the "smallest" ellipse in fii is with semimajor axis and semiminor axis 2^y\|3l|32\ and 
0, respectively. The "largest" ellipse in Q2 is with semimajor axis and semiminor axis 
|1 + Pi/32\ and \Pi(32 — 1|, respectively; the "smallest" eUipse in Q2 is with semimajor axis 
and semiminor axis and ^^^r^fr^- So both ri-i and Q2 are the closure of the domain 
enclosed by the ellipse with center the origin point and semimajor axis and semiminor axis 
|1 + /3i/32| and \P1P2 ~ respectively. Thus fli = fl2. □ 



4 R-diagonal operators in Ai 

In this section, we prove the following result. We will use the notation introduced in section 
2.4. 

Theorem 4.1. In M, let A e M2(C)(i) and B e M2(C)(2). Then AB is an R-diagonal 
operator if and only ifT{A) — t{B) — 0. 

To prove Theorem 4.1, we need the following lemmas. 

Lemma 4.2. {Wi, V^, W^V^}" ^ L(Z2) * L(Z2) * L(Z). 

Proof. Let U — Then [/ is a Haar unitary operator. We need only to prove that 

[/ is * free with the von Neumann subalgebra generated by Wi and Vi. Let gig2 • • • g'n be 
an alternating product of : n 7^ 0} and {Wi, W^Vi, ViWi, WiViWi, ViWiVi, ■■■}. 
By regrouping, it is an alternating product of {Wi, WiW^, W^Wi, W^WiW^, H/3, W^] and 
{Vu V^Vi, V^ViVi, V?„ V^}. Thus the trace is 0. □ 

Lemma 4.3. \ ^ '^rl ] \ ^ ] is an R-diagonal operator. 



A ;a)V/^2 



Proof. Note that 

Oai\ [ Q a2\ [ ai Q \ /Ol\ /Ol\ f [^2 



A 0y(i)V/52 y^,) \0 ^3^J^,^\1 0J^^^\1 0J^^^\0 a2,^,^ 

By Lemma 4.2 and basic properties of R-diagonal operators given in 2.2, we prove the 
lemma. □ 
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Lemma 4.4. With the assumption of Theorem 4-1 (ind assume AB is an R-diagonal 
operator and t{A^) ^ 0. Then t{B) = 0. 

Proof. Since AB is an i?— diagonal operator, t{AB) = 0. Since A, B are *-free, t{A)t{B) = 
t{AB) = 0. If t{B) = 0, then done. Otherwise, assume t{A) = 0. Then = t{ABAB) = 
tIa^B)t{B) = T{A^){T{B)f. By assumption, t{B) = 0. □ 

Lemma 4.5. Let B G M2(C)(2) and X be any complex number. Then a{Ei2B) = a{Ei2{X + 
B)). 

Proof. By Jacobson's theorem, 

c7(£;i2(A + B)) U {0} = a(En^i2(A + B)) U {0} = a(Ei2(A + B)E^^) U {0} 
= a{Ei2BEi^) U {0} = cr{BEu) U {0}. 

□ 

Proof of Theorem 4-T If t{A) = t{B) = 0, then by Lemma 3.3 and Lemma 4.3, AB 
is an R-diagonal operator. Conversely, assume that AB is an R-diagonal operator. Then 
= t{AB) = t{A) ■ t{B). So either t{A) = or t{B) = 0. Without loss of gen- 
erality, we assume that t{A) = 0. If T(y4^) ^ 0, then t{B) = by Lemma 4.4. If 
tIA"^) = 0, then A is unitary equivalent to aEi2- We may assume that A = E12. By 
Theorem 2.1, if E12B is an R-diagonal operator, then {r{Ei2B)Y = t{B* E21E12B) = 
t{E2iEi2BB*) = WEuWl ■ \\B\\l. Note that ^12(5 - t{B)) is an R-diagonal operator, 
ir{E,2{B - T{B))f = \\E,2\\l ■ \\B - t{B)\\1 By Lemma 4.5, \\B\\l = \\B - t{B)\\1 This 
implies that t{B) = 0. This ends the proof. 

Combining Theorem 4.1, Theorem 2.1 and § transform of Voiculescu (see flQ[ [T7]). we 
have the following theorem (It is interesting to compare the following theorem and Theorem 
3.6). 

Theorem 4.6. Let A G Af2(C)(i),5 G Af2(C)(2) and t{A) = t{B) = 0. Then 

1. fiAB is rotation invariant; 

2. a{AB) = suppfiAB = [\\A-'\\2'\\B-'\\2\ Plhll^lb] [0,27r]; 

3. fiAsiiO}) = max{/iA*yi({0}), /iB*B({0})} and 

/iAB(B(0, (§^,., WJt - 1))-^/')) = t, for t G !]• 
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5 Algebraic techniques 

For X e M, define 

y E12XE11 E12XE21 

Then $ is a *-isomorphism from Ai onto EuAiEn ® M2(C)(i). We will identify Ai with 
EiiAiEii M2(C)(i) by the canonical isomorphism $. In [1], K. Dykema proved that 
EiiMEu = L{¥s). For B e M2(C)(2), we may write 



B 



hi bi2 

^21 ^22 



with respect to matrix units in Aii. Then bij G /^(Fs). In this section, we will develop 
the algebraic techniques used in [1]. Combining the matrix techniques, we may explicitly 
express bij in terms of free generators of L(F3). 

Let A{pyi, V"i} be the set of words generated by Wi,Vi. Note that Wf = = 1 and 
t{Wi) = t(Vi) = 0. The following observation is crucial in [4j. The proof is an easy 
exercise. 

Lemma 5.1. T{gig2 - ■ ■ Qn) = for an alternating product of A{Wi,Vi} \ {l,Wi} and 
{E12, E21}. 

Recall that P = En and Q = Fu. Let W be the "polar" part of (1 - P)QP and 
U = E12W . The following corollary is a special case of Theorem 3.5 of 

Corollary 5.2. U is a Haar unitary operator in M.p = PM.P and U , PQP are *-free in 
Mp. 

With the canonical identification of Ai with Aip ® M2(C)(i), 

Q = 



PQP ^PQP - {PQPyu 

U*^PQP- {PQPy U*{1~PQP)U 

By [H], the distribution of PQP (relative to A4p) is non-atomic and the density function 
is 

Pit) = - , ^ < t < 1. (5.1) 

By Corollary 15. 2[ the von Neumann subalgebra Aii generated by M2(C)(i) and Q is 
^-isomorphic to L(¥2) M2(C)(i). Since Aii is also ^-isomorphic to M2(C) * L{Z2), 
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M2(C) * L(Z2) = L(F2) ® M2(C), which is proved by Dykema in [1]. 



Since Vi = 2Q - I, 




2PQP - 1 
2U*^PQP -{PQPy 



2^PQP -{PQpyu 

U*{l-2PQP)U 



Simple computation shows that the density function of 2PQP — 1 is 



1 



1 



71 



~i<t<i. 



Let H = 2PQP - 1, then 



H VI - H^U 

U*^l-m -U*HU 



) 



Let H = V\H\ be the polar decomposition of H. Since if is a symmetric selfadjoint 
operator, V"^ = 1 and V is independent with the von Neumann algebra generated by \H\ in 
the classical probability sense. Let h = \H\,u = VU,v = UV. Then u,v are Haar unitary 
operators and the distribution of h relative to A^p is non-atomic. 

Lemma 5.3. h,u,v are * free. 

Proof. Let gig2 ■ ■ ■ (?„ be an alternating product of elements of & = {\H\}" Q CI, {(VU)"' : 
n 7^ 0}, {{UV)" : n 7^ 0}. By regrouping, it is an alternating product of elements of 
{©, V, V&, &V, VeV} and {f/" : n 7^ 0}. Since H and U are *-free, {&, V, V&, 6V, VeV} 
and : n 7^ 0} are free. Since V and & are independent, t{VS) = t{SV) = for 5 G 6. 
This implies that T{gig2 ■ ■ ■ gn) = 0- 



□ 



By simple computations, we have the following. 




(5.2) 




(5.3) 




(5.4) 



13 



By Lemma 2.2, M = M2(C)(i) * (ViM2(C)(i) Vi) = Mp ® M2(C)(i). With this isomor- 
phism, Alp is the von Neumann algebra generated by h,u and v by fl5.2p and fl5.4p . So 
J\Ap = ^(Fs). By simple computations, we have 

Y ( 1^ \ ^ ^ Z' &11 ^12 

V 7 / (1) ^ V ^21 &22 

where 



6i2 = (a - a)h^/l - /i^M + 7VI - h^v\/l - h^u - (3hv*hu, 

621 = (a - a)u*hVT^n? - 7M*/it;/i + /?mV1 - /iVVl - h^, 

622 = a + (cr — a)u*h'^u — ■yu*hvy/l — K^u — (3u*\/l — h'^v*hu. 

Theorem 5.4. A^=L(F3) ® M2(C)(i); furthermore, let B = ( ^ ^ \ m M2(C)(2), 

V ^ ^ / (2) 

, , ) , where 

are given as above. 



Example 5.5. In Theorem 15.41 let /3 = 7 = 0. Then we have 



a \ _ / a + {a — a)h'^ {a — a)h^/l — 



u 



^ J (^2) V ('^ ~ a)u*hy/l — a + (a — a)u*h^u j 



Example 5.6. In Theorem 15. 4[ let a = cr and 7 = 0. Then we have 



a fi\ _( a + Phv*y/1- h'^ -phv*hu 
a 7(2)" V PuWl- h^vWl - a- (3uWl- h^v*hu J 

Remark 5.7. By equation (15. 2p . the distribution of h'^ is the distribution of EuViEuViEu 
relative to Mp. So the distribution of h'^ is same as the distribution of PQP (relative to 
Aip). By [Vo], the distribution of PQP (relative to Aip) is non-atomic and the density 
function is 

p(t) = - , ^ < ( < 1. 
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6 Miscellaneous examples 

Example 6.1. We compute the Brown spectrum of aEi2+PFi2. Let F12 ~ ^ g ) 
. Then 



(2) 



bi 62 
63 64 



(1) 



{aEi2 + pFi2f = ap{Ei2Fi2 + F12E12) = ap{Eu + Fi2f = 



63 hi + 64 
63 



(1) 



So /i(a£;i2+/3Fi2)2 = fJ'a/sba- By equatiou (15.31) . the distribution of 63 is same as the distribution 
of (f/* Vl — H'^y. Since U*\/l~ H'^ is an R-diagonal operator, {U*\/l — H'^)^ is also an R- 
diagonal operator. Since the distribution of aEi2 + (3Fi2 is rotation invariant, fiaEi2+i3Fi2 = 
f^y/\a^\b^ where b = U*y/1 — H'^. Simple computations show that (or by Proposition 5.10 
and Corollary 5.11 of [5j) 

di^b{z) = -—, — ^-^^drdO < r < 
TT (1 - r2)2 

Hence 



1 llaPl 



and 




Corollary 6.2. r{aEu + PFu) = y'^. 

Corollary 6.3. Let A e M2(C)(i) and B G M2(C)(2). Then A + B is an R-diagonal 
operator if and only if A + B = 0. 

Proof. Indeed, if A + i? is an R-diagonal operator, then t{A + B) = 0. So we may assume 
that t{A) = t{B) = 0. Let A, —A and rj, —r] be the spectra of A and B, respectively. 
Then = t{{A + B^) = r(A^) + t{B'^) = A^ + 77^. By simple computation we have = 
r((A + 5)4) = t{A^)+t{B^) + At{A^)t{B^) = AHr/H4AV = {X^ + V^ + '^^W = 2AV- 
Thus A = ?7 = 0. This implies that A and B are unitary equivalent to aEi2 and (3Fi2, 

respectively. By Corollary 16.21 r{A + B) = ^-1^. On the other hand, since we assume that 

A + B is an R-diagonal operator, by Theorem 2.1, {r{A + B)Y = t{{A* + B*){A + B)) = 

T{{aE2i + pF2i){aEi2 + PF^)) = So |ap + = \a(3\. This implies that a = 

and p = 0. Hence A + B = 0. □ 
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Example 6.4. We compute the spectrum and Brown spectrum of 

1 ^ ^ a p 

(i)VO «/(2) 

By Example 15.61 we have the following 



^ ■ y V a 



X 



10\ fa p\ /10\ / a + phv*Vl - -phv*hu 



y (1) V " /(2) VO J f3u*VT^vVT^ a- pu*VT^v*hu 



a + phvWl - ~phv*hu 




(1) 

So(t(X) = {0}Ua{a+phv*y/T^ni^) and nx = |'^o + |/Ua+/3to.v'T3^- Note that /i/^^.^/j^Ti? = 
f^v*Vi^h v*Vl — h^h is an R-diagonal operator. We have the following computations: 

1 t{l - t)dt 1 



||v/r^/i||^ = rp((l - h'')h^) = rp((l - PQP)PQP) = [ 

Jo 



\\{vrn^h)-x = - h')hy) = - pqp)pqp)-') 

1 dt 



oo. 



''t{^-t)\\-{\-tY 



By Theorem 2.1, 



(t{X) = supp^x = 
Example 6.5. We compute the spectrum and Brown spectrum of 

^ / 1 A / a 

By Example 15.51 we have the following 
Y = 



01\ fa 0\ _/0l\ / P+{a-pyf_ {a-p)hVl^^u 



a - (3)u*h\/\ - a + {(3 - a)u*h'^u 



(1) 



Since u*h\/l — is an R-diagonal operator, similar computations as Example 16.41 we have 



{Y) = suppfiY = M{0,\a- P\/2V2). 



16 



Example 6.6. We compute the spectrum and Brown spectrum of 



Z = (l + aEi2)(l + /?Fi2) 



1 a\ ( I 13 



1 ^i)VO 1.(2) 



For A G C, we have 

Z-\l = (l + aEi2)(l+/3Fi2)-A(l+aEi2)(l-aEi2) = {l + aEu){XaEu + PFi^- (X-l))- 



This imphes that A G o-(Z) if and only if A — 1 G a{XaEi2 + (3Fi2). By Example I6.H 
A — 1 G cr(Aai?i2 + I3F12) if and only if 



|A-lp< 

So 



|a/3||A| 



(7(z) = n G c : |A- ir < 



In the following, we will show that suppfiz ^ dcr^Z) = |a G C : |A — Ip = ^"^^^'^^ |. For 
this purpose, we need only to prove that suppfiz-i ^ da{Z—l) = |a G C : |Ap = 

Note that A(l + 0^12) = 1. For A G C, we have 

log A((Z - 1) - A) = log A((l + aEi2){l + f^Fu) - (1 + A)(l + a£;i2)(l - aEu)) 
= log A(l + aEi2) +logA(l + /3Fi2 - A + (1 + X)aEu) = logA((l + X)aEi2 + PFi2 - A). 
By Example O, /i(i+A)a£;i2+/3Fi2 = f^^ \i+x\\a(3\ b- Hence, 

log A((Z - 1) - A) = log A(v/|l + X\\a(3\b - A) 

= logA I b- ^ ^ =] -log , *^* ^ + log|A|. 
Since b is an R-diagonal operator, this implies that 

log A f 6 - ) = log W - log |A| + log A((Z - 1) - A). (6.1) 

Suppose Ao G da{Z — 1) and Aq ^ suppfiz^i- Then there is 5 > such that B(Ao, S) C C\ 
suppfiz-i- Now log A((Z— 1) — A) is a harmonic function on B(Ao, 5). Since r((Z — 1)") =0 
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for all n = 1,2,- ■■. By Lemma 4.3 of [8j, for A G C such that |A| > r{Z-l), logA((Z-l)- 
A) = log I A|. By the uniqueness of harmonic functions, we have log A((Z — 1) — A) = log |A| 
for A G B(Ao,5). By equation fl6.ip . this implies that 

log A Ib-^ I =log , . (6.2) 

Let r = . Then equation (16. 2p implies that 

y/\l+X\\al3\ 

log A (6 — r) = log r 

for r G (s,t) C [0, -^]. Since b is an R-diagonal operator, this implies that log A(6 — z) 
is harmonic on the annulus with inner radius s and outer radius t, < s < t < By 
Theorem 2.1, suppfib = B(0, ^). It is a contradiction. 

7 Hyperinvariant subspaces for operators in A4 

Lemma 7.1. For X E Ai, if suppfix = {A}, then t(X") = A" for n = 1,2, ■ ■ ■ . 

Proof r(X") = r ;z'^dux(^) = A'^. □ 

The converse of Lemma 17.11 is not true. Since for an R-diagonal operator X, we have 
r(X") = for n = 1, 2 ■ ■ ■ . 

Proposition 7.2. LttX = A^B, where A G M^iC)^^) and B G M2(C)(2). If A,B are 
not scalar matrices, then suppfix contains more than two points. 

Proof. Suppose A, B are not scalar matrices. Since A + B = t{A)1 + t{B)1 + {A — 
t{A)1) + {B — t{B)1), to show supp^x contains more than two points, we need only to 
show fi{A~T{A)i)+{B-T{B)i) contains more than two points. So we may assume that t{A) = 
r{B) = and A,B 0. Assume that the spectra of A and B are Ai, — Ai and A2, — A2. If 
t{A'^) = t{B^) = 0, then A and B are unitarily equivalent to aEi2 and PF12 in M2(C)(i) 
and M2(C)(2), respectively. Thus fix = l^'aEu+l^Frz- By Example 16. H suppnx contains more 
than two points. Now suppose t{A^) 7^ or t^B"^) 7^ 0. Without loss of generality, we 
assume that \\ = t{A^) ^ 0. Note that t{A + B) = and T{{A + Bf) = t{A^)+t{B^) = 
Xf + X^. If A^ + A2 7^ 0, by Lemma mi suppnx contains more than two points. Suppose 
A^ + A2 = 0. Then ri^B"^) = Aj 7^ 0. Simple computations show that 

T{{A+Bf) = T{A')+T{B^)+AT{A^)r{B^) = At+A^+4A?A^ = {\l+\lf +2X1x1 = 2X1X1 ^ 



Note that t[A + i?) = 0. By Lemma [7. H suppnx contains more than two points. □ 
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Proposition 7.3. Let X = AB , where A e M2(C)(i) and B G M2(C)(2). If A,B are not 

scalar matrices, then suppfix contains more than two points. 

Proof. Suppose A, B are not scalar matrices. We consider the following cases: 

Case 1. t{A) = t{B) = and A,B ^ 0. By Theorem 4.1, AB{^ 0) is an R-diagonal 
operator. So suppfix contains more than two points. 

Case 2. t{A) = 0, t{B) 7^ or t{A) ^ 0, t{B) = 0. Without loss of generality, we assume 
that t{A) = and t{B) =^ 0. Then t{AB) = and t{ABAB) = T{A^)r{B). If t{A^) ^ 0, 
then t{ABAB) 7^ 0. By Lemma 10.1, suppfix contains more than two points. If t{A'^) = 0, 
then A is unitarily equivalent to aEi2 in M2(C)(i). By Lemma 4.5, fix = f^aEi^iB-riB))- 
Since aEi2{B — t{B){^ 0) is an R-diagonal operator, suppfix contains more than two 
points. 

Case 3. t{A) ^ and t{B) ^ 0. We may assume that t{A) = t{B) = 1. Let A = 1 + Ai 
and S = 1 + Bi. Then r(Ai) = r(5i) = 0. 

Subcase 3.1. t{A1) 7^ or T{Bf) 7^ 0. We may assume that t{A1) 7^ 0. Simple computa- 
tion shows that r(A5) = 1, t{ABAB) = 1 + t{AI)+t{BI) and T{{ABf) = l + 3{T{Aj) + 
t{BI)) + 9t{AI)t{BI). lfT{Al)+T{Al) ^ 0, then t{ABAB) ^ 1. By LemmaEH suppfix 
contains more than two points. If t{AI) + t{AI) = 0, then t{AI) = —t{AI) 7^ 0. So 
t{{AB)^) 7^ 1. By Lemma [7. II again, suppfix contains more than two points. 

Subcase 3.2. t{A^) = t^A"^) = 0. Then Ai and A2 are unitarily equivalent to aE^ and 
PFi2 in M2(C)(i) and M2(C)(2), respectively. So fix = fJ'(i+aEi2){i+/3Fi2)- We may assume 

that A = ( ^ ^ ] and B = ( ^ ^ ] . By Example 16.61 suppfix contains more than 



two points. □ 

Corollary 7.4. Let X = AB or X = A + B, where A G M2(C)(i) and B G M2(C)(2). // 
X 7^ AI, then X has a nontrivial hyperinvariant subspace relative to Ai. 

Proof li X = A + B and A = \1 or B = XI, then X = \1 + B or X = \1 + A. If X is not 
a scalar matrix and r] is an eigenvalue of X, then ker{X — r]l) is a nontrivial hyperinvariant 
subspace of X. li X = A + B and A,B XI, then suppfix contains more than two points 
by Proposition 17.31 By [9J, X has a nontrivial hyperinvariant subspace relative to Ai. If 
X = AB and A = XI or B = XI, then X = XB or X = XA. If X is not a scalar matrix 
and rj is an eigenvalue of X, then ker{X — rjl) is a nontrivial hyperinvariant subspace of 
X. If X = AB and A, B ^ XI, then suppfix contains more than two points. By [9], X has 
a nontrivial hyperinvariant subspace relative to Ai. □ 
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